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DETAILED ACTION 
Response to Amendment 

1 . This Office action is responsive to the Request for Continued Examination (RCE) 
filed under 37 CFR §1 .53(d) on 1/30/06. Applicants have properly set forth the RCE, 
which has been entered into the application, and an examination on the merits follows 
herewith. The Examiner acknowledges Applicant's amendments to claims 1, 29 and 56 
to better specify the current invention. All pending claims have been rejected as being 
obvious over prior art disclosed. 

Claim Rejections - 35 USC § 103 
The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

2. Claims 1-7, 9-34, 36-56 and 58-86 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over "Adjustable Filmstrips and Skims as Abstractions for a Digital Video 
Library" (Christel et al.). herein referred to as Christel and U. S. Patent No. 6, 880, 171 
B1 (Ahmad et a!.), herein referred to as Ahmad. 

Referring to claims 1 and 2, Christel discloses a method of presenting 
information regarding a video with a plurality of frames (page 2, column 2, lines 1 1-19). 
Christel discloses summarizing a video by identifying a plurality of segments of the 
video based upon an event (page 4, column 2, lines 28-32). Christel discloses that 
each of the segments includes a plurality of sequential frames of the video (page 4, 
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column 2, lines 8-20). Christel discloses displaying a summarization in a first portion of 
a display (Figure 5), wherein the first portion discloses a snapshot of a skim and 
indicating the sequences that make up the skim at the bottom of the window. Christel 
discloses displaying a graphical user interface on a second portion of the display, 
sequentially indicating the relative location of the plurality of segments within the 
summarization relative to at least one other of the segments as each of the plurality of 
segments is displayed (Figure 5), wherein the second portion displays the match 
locations in relation to the location of plurality of segments within the summarization. 
Christel discloses displaying the relative location for a first type of content of the video 
using a first visual indication (Figure 5), wherein this is represented as the gray blocks 
at the bottom of the window and displaying the relative location for second type of 
content of the video using a second visual indication different from the first visual 
indication (Figure 5), wherein the white lines showing the match locations represents 
the second visual indication. Christel does not explicitly disclose that the event being 
captured is characterized by a play, wherein this play is of a sport. It would have been 
obvious for one skilled in the art at the time of the invention that the event being 
captured is characterized by a play within a sport. Christel includes all types of video 
when discussing capturing of important events from a video, wherein based on the 
content of the video, any important events relevant to that specific video would be 
captured. Christel discloses that the goal of the invention is to display the essential 
content of a video, and wherein it is obvious that when the video is that of a sporting 
event, the essential content would be the plays and hence it would be obvious for 
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Christel to captures these plays, wherein these plays are those that are part of a 
sporting event. Christel does not disclose two different types of semantic events 
through indicated by two different visual means. Ahmad also discloses obtaining 
summary data related to video and audio data, with visual means for indicating a 
relative location for a first type of semantic event in a video data and displaying the 
relative location for a second type of semantic event in the video using a second visual 
indication different from the first visual indication (column 16, lines 54-65). It would 
have been obvious for one skilled in the art, at the time of the invention to learn from 
Ahmad to teach two different visual indications for two different types of semantic event 
information. Both Ahmad and Christel disclose systems for summarizing and obtaining 
relevant data from video and audio data, both further displaying graphical bars for 
indicating relevant data. Ahmad further displays detailed bar information to indicate 
different visual means for different types of semantic event data. It would have been 
obvious for one skilled in the art, at the time of the invention to learn from Ahmad to 
implement detailed bar data to further provide more detailed information concerning the 
video and summary data obtained. 

Referring to claim 3, Christel discloses a generally rectangular region where each 
of the plurality of segments is indicated within the generally rectangular region (Figure 
5). 

Referring to claim 4, Christel discloses that the size of each of the plurality of 
segments is indicated in a manner such that the plurality of segments with a greater 
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number of frames are larger than the plurality of segments with a lesser number of 
frames (Figure 5). 

Referring to claim 5, Christel discloses that the size of the regions between each 
of the plurality of segments is indicated in a manner such that the regions between with 
a greater number of frames are larger than plurality of segments with a lesser number of 
frames (Figure 5). 

Referring to claim 6, Christel discloses an indicator that indicates the current 
position within the temporal information of a currently displayed portion of the video 
(Figure 5). 

Referring to claim 7, Christel discloses the indicator changes location relative to 
the temporal information as the portion of the currently displayed portion of the video 
changes (Figure 5). 

Referring to claim 9, Christel discloses indicating with an indicator the current 
position within the temporal information of a currently displayed portion of said video 
and modifying the position of the indicator within the temporal information, which 
modifies the displayed portion of the video (Figures 5 and 6). 

Referring to claim 10, Christel discloses that the indicator is modified to a portion 
of the video that is not included within the plurality of segments (Figures 5 and 6). 

Referring to claim 11, Christel discloses the first type of content and second type 
of content are selectable by a user for presentation to the user (Figure 5, page 5, 
column 1, lines 9-12 and Figure 6). 
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Referring to claim 12, Christel discloses that at least two of the plurality of 
segments is temporally overlapping (page 4, column 2, lines 26-29). 

Referring to claim 13, Christel discloses that the temporally overlapping 
segments are visually indicated in a manner such that each of the overlapping 
segments is independently identifiable (page 4, column 2, lines 26-31). 

Referring to claim 14, Christel discloses that a user selects a portion of the video 
not included within the plurality of segments, wherein in response thereto, the system 
presents one of the plurality of segments (Figure 6). 

Referring to claim 15, Christel discloses that one of the plurality of segments is 
the segment most temporally adjacent to the portion of the video (Figure 6). 

Referring to claim 16, Christel discloses that one of the plurality of segments is 
the next temporally related segment (Figure 6). 

Referring to claim 17, Christel discloses that the plurality of segments is the 
previous temporally related segment (Figure 6). 

Referring to claim 18, Christel discloses that a user selects a portion of the video 
included within the plurality of segments, wherein in response thereto, the system 
presents the portion of the video from the start thereof (page 5, column 1 , lines 5-11). 

Referring to claim 19, Christel discloses that a user selects a portion of the video 
not included within the plurality of segments, wherein in response thereto, the system 
presents one of the plurality of segments, and wherein the user selects a portion of the 
video included within the plurality of segments, wherein in response thereto, the system 
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presents the portion of the video within the plurality of segments (page 5, column 1 , 
lines 5-11 and Figure 6). 

Referring to claim 20, Christel discloses that a user selects a portion of the video 
not included within the plurality of segments, wherein in response thereto, the system 
presents one of the plurality of segments, and wherein the user selects a portion of the 
video included within the plurality of segments, wherein in response thereto, the system 
presents the portion of the video within the plurality of segments starting from the 
beginning thereof (page 5, column 1, lines 5-11 and Figure 6), 

Referring to claim 21 , Christel discloses that a user selects a portion of video not 
included within the plurality of segments, wherein in response thereto, the system 
presents the selected portion not Included within the plurality of segments, and wherein 
after presenting the selected portion not included within the plurality of segments 
presents the selected plurality of segments in temporal order without portions of the 
video not included within the plurality of segments, and wherein the user selects a 
portion of the video included within the plurality of segments, wherein in response 
thereto, the system presents the portion of the video within the plurality 
of segments (page 5, column 1, lines 5-11 and Figure 6). 

Referring to claim 22, Christel discloses that the temporal information is 
hierarchical and is displayed in such a manner to retain a portion of its hierarchical 
structure (Figure 6). 

Referring to claim 23, Christel discloses that temporal information relates to 
overlapping time periods and the temporal information is displayed in such a manner to 
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maintain the differentiation of the overlapping time periods (Figures and page 4, column 
2, lines 26-31). 

Referring to claim 24, Christel discloses that the temporal information is 
displayed within a time line, wherein the temporal information is presented in a plurality 
of layers in a direction perpendicular to the length of the time line (Figure 5). 

Referring to claim 25, Christel discloses that the temporal information is 
displayed within a time line, wherein textual information is included within the time line 
(Figure 6). 

Referring to claim 26, Christel discloses that additional textual information is 
displayed upon selecting a portion of the time line (Figure 6). 

Referring to claim 27, Christel discloses that temporal information is displayed 
together with a time line, wherein additional textual information is displayed together 
with selecting a portion of the time line (Figure 6). 

Referring to claim 28, Christel discloses that temporal information is displayed 
within a time line, wherein additional audio annotation is presented upon presenting a 
portion of the time line (page 4. column 1, lines 33-37 and Figure 6). 

Referring to claim 29, Christel discloses a method of presenting information 
regarding a video with a plurality of frames (page 2, column 2, lines 11-19). Christel 
discloses identifying a plurality of different segments of the video, where each of the 
segments includes a plurality of frames of the video (page 4, column 2, lines 8-32). 
Christel discloses displaying, simultaneously with a segment of the video, a graphical 
user interface including information regarding the temporal location of the segments. 
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relative to at least one other of the segments of the video (Figure 5). Christel discloses 
displaying the temporal location for a first type of content of the video using a first visual 
indication (Figure 5), wherein this is represented as the gray blocks at the bottom of the 
window and displaying the temporal location for second type of content of the video 
using a second visual indication different from the first visual indication (Figure 5), 
wherein the white lines showing the match locations represents the second visual 
indication. Christel does not disclose two different types of semantic events through 
indicated by two different visual means. Ahmad also discloses obtaining summary data 
related to video and audio data, with visual means for indicating a relative location for a 
first type of semantic event in a video data and displaying the relative location for a 
second type of semantic event in the video using a second visual indication different 
from the first visual indication (column 16, lines 54-65). It would have been obvious for 
one skilled in the art, at the time of the invention to learn from Ahmad to teach two 
different visual indications for two different types of semantic event information. Both 
Ahmad and Christel disclose systems for summarizing and obtaining relevant data from 
video and audio data, both further displaying graphical bars for indicating relevant data. 
Ahmad further displays detailed bar information to indicate different visual means for 
different types of semantic event data. It would have been obvious for one skilled in the 
art, at the time of the invention to learn from Ahmad to implement detailed bar data to 
further provide more detailed information concerning the video and summary data 
obtained. 
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Referring to claim 30, Christel discloses a generally rectangular region where 
each of the plurality of segments is indicated within the generally rectangular region 
(Figure 5). 

Referring to claim 31 , Christel discloses that the size of each of the plurality of 

segments is indicated in a manner such that the plurality of segments with a greater 

number of frames are larger than the plurality of segments with a lesser number of 

« 

frames (Figure 5). 

Referring to claim 32, Christel discloses that the size of the regions between 
each of the plurality of segments is indicated in a manner such that the regions between 
with a greater number of frames are larger than plurality of segments with a lesser 
number of frames (Figure 5). 

Referring to claim 33, Christel discloses an indicator that indicates the current 
position within the temporal information of a currently displayed portion of the video 
(Figure 5). 

Referring to claim 34, Christel discloses the indicator changes location relative to 
the temporal information as the portion of the currently displayed portion of the video 
changes (Figure 5). 

Referring to claim 36, Christel discloses indicating with an indicator the current 
position within the temporal information of a currently displayed portion of said video 
and modifying the position of the indicator within the temporal information, which 
modifies the displayed portion of the video (Figures 5 and 6). 
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Referring to claim 37, Christel discloses that the indicator is modified to a portion 
of the video that is not included within the plurality of segments (Figures 5 and 6). 

Referring to claim 38, Christel discloses the first type of content and second type 
of content are selectable by a user for presentation to the user (Figure 5, page 5, 
column 1 , lines 9-12 and Figure 6). 

Referring to claims 39 and 60, Christel discloses that at least two of the plurality 
of segments Is temporally overlapping (page 4, column 2, lines 26-29). 

Referring to claims 40 and 61 , Christel discloses that the temporally overlapping 
segments are visually indicated in a manner such that each of the overlapping 
segments is Independently identifiable (page 4, column 2, lines 26-31). 

Referring to claim 41, Christel discloses that a user selects a portion of the video 
not included within the plurality of segments, wherein in response thereto, the system 
presents one of the plurality of segments (Figure 6). 

Referring to claim 42, Christel discloses that one of the plurality of segments is 
the segment most temporally adjacent to the portion of the video (Figure 6). 

Referring to claim 43, Christel discloses that one of the plurality of segments is 
the next temporally related segment (Figure 6). 

Referring to claim 44, Christel discloses that the plurality of segments is the 
previous temporally related segment (Figure 6). 

Referring to claim 45, Christel discloses that a user selects a portion of the video 
included within the plurality of segments, wherein in response thereto, the system 
presents the portion of the video from the start thereof (page 5, column 1 , lines 5-1 1 ). 
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Referring to claim 46, Christel discloses that a user selects a portion of the video 
not included within the plurality of segments, wherein in response thereto, the system 
presents one of the plurality of segments, and wherein the user selects a portion of the 
video included within the plurality of segments, wherein in response thereto, the system 
presents the portion of the video within the plurality of segments (page 5, column 1, 
lines 5-1 1 and Figure 6). 

Referring to claim 47, Christel discloses that a user selects a portion of the video 
not included within the plurality of segments, wherein in response thereto, the system 
presents one of the plurality of segments, and wherein the user selects a portion of the 
video included within the plurality of segments, wherein in response thereto, the system 
presents the portion of the video within the plurality of segments starting from the 
beginning thereof (page 5, column 1, lines 5-11 and Figure 6). 

Referring to claim 48, Christel discloses that a user selects a portion of video not 
included within the plurality of segments, wherein in response thereto, the system 
presents the selected portion not included within the plurality of segments, and wherein 
after presenting the selected portion not included within the plurality of segments 
presents the selected plurality of segments in temporal order without portions of the 
video not included within the plurality of segments, and wherein the user selects a 
portion of the video included within the plurality of segments, wherein in response 
thereto, the system presents the portion of the video within the plurality of segments 
(page 5, column 1 , lines 5-1 1 and Figure 6). 
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Referring to claim 49, Christel discloses that the temporal information is 
hierarchical and is displayed in such a manner to retain a portion of its hierarchical 
structure (Figure 6). 

Referring to claim 50, Christel discloses that temporal information relates to 
overlapping time periods and the temporal information is displayed in such a manner to 
maintain the differentiation of the overlapping time periods (Figures and page 4, column 
2, lines 26-31). 

Referring to claims 51 and 72, Christel discloses that the temporal information is 
displayed within a time line, wherein the temporal information is presented in a plurality 
of layers in a direction perpendicular to the length of the time line (Figure 5). 

Referring to claims 52 and 73, Christel discloses that the temporal information is 
displayed within a time line, wherein textual information is included within the time line 
(Figure 6). 

Referring to claims 53 and 74, Christel discloses that additional textual 
information is displayed upon selecting a portion of the time line (Figure 6). 

Referring to claims 54 and 75, Christel discloses that temporal information is 
displayed together with a time line, wherein additional textual information is displayed 
together with selecting a portion of the time line (Figure 6). 

Referring to claims 55 and 76, Christel discloses that temporal information is 
displayed within a time line, wherein additional audio annotation is presented upon 
presenting a portion of the time line (page 4, column 1, lines 33-37 and Figure 6). 
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Referring to claim 56, Christel discloses a method of presenting information 
regarding audio (page 2, column 2, lines 11-19). Christel discloses identifying a plurality 
of different segments of the audio, where each of the segments includes a temporal 
duration of the audio (page 4, column 2, lines 8-32). Christel discloses displaying, 
simultaneously with the segment of the audio, a graphical user interface including 
information regarding the temporal location of the segments, relative to at least one 
other of segment of the audio (Figure 5). Christel discloses displaying the temporal 
location for a first type of content of the audio using a first visual indication (Figure 5), 
wherein this is represented as the gray blocks at the bottom of the window and 
displaying the temporal location for second type of content of the audio using a second 
visual indication different from the first visual indication (Figure 5), wherein the white 
lines showing the match locations represents the second visual indication. Christel 
does not disclose two different types of semantic events through indicated by two 
different visual means. Ahmad also discloses obtaining summary data related to video 
and audio data, with visual means for indicating a relative location for a first type of 
semantic event in a video data and displaying the relative location for a second type of 
semantic event in the video using a second visual indication different from the first 
visual indication (column 16, lines 54-65). It would have been obvious for one skilled in 
the art, at the time of the invention to learn from Ahmad to teach two different visual 
indications for two different types of semantic event information. Both Ahmad and 
Christel disclose systems for summarizing and obtaining relevant data from video and 
audio data, both further displaying graphical bars for indicating relevant data. Ahmad 
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further displays detailed bar information to indicate different visual means for different 
types of semantic event data. It would have been obvious for one skilled in the art, at 
the time of the invention to learn from Ahmad to implement detailed bar data to further 
provide more detailed information concerning the video and summary data obtained. 

Referring to claim 58, Christel discloses indicating with an indicator the current 
position within the temporal information of a currently displayed portion of said audio 
and modifying the position of the indicator within the temporal information, which 
modifies the displayed portion of the audio (Figures 5 and 6). 

Referring to claim 59, Christel discloses that the indicator is modified to a portion 
of the audio that is not included within the plurality of segments (Figures 5 and 6). 

Referring to claim 62, Christel discloses that a user selects a portion of the audio 
not included within the plurality of segments, wherein in response thereto, the system 
presents one of the plurality of segments (Figure 6). 

Referring to claim 63, Christel discloses that one of the plurality of segments is 
the segment most temporally adjacent to the portion of the audio (Figure 6). 

Referring to claim 64, Christel discloses that one of the plurality of segments is 
the next temporally related segment (Figure 6). 

Referring to claim 65, Christel discloses that the plurality of segments is 
the previous temporally related segment (Figure 6). 

Referring to claim 66, Christel discloses that a user selects a portion of the audio 
included within the plurality of segments, wherein in response thereto, the system 
presents the portion of the audio from the start thereof (page 5, column 1 , lines 5-11); 
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Referring to claim 67, Christel discloses that a user selects a portion of the audio 
not included within the plurality of segments, wherein in response thereto, the system 
presents one of the plurality of segments, and wherein the user selects a portion of the 
audio included within the plurality of segments, wherein in response thereto, the system 
presents the portion of the audio within the plurality of segments (page 5, column 1, 
lines 5-11 and Figure 6). 

Referring to claim 68, Christel discloses that a user selects a portion of the audio 
not included within the plurality of segments, wherein in response thereto, the system 
presents one of the plurality of segments, and wherein the user selects a portion of the 
audio included within the plurality of segments, wherein in response thereto, the system 
presents the portion of the audio within the plurality of segments starting from the 
beginning thereof (page 5, column 1, lines 5-1 1 and Figure 6). 

Referring to claim 69, Christel discloses that a user selects a portion of audio not 
included within the plurality of segments, wherein in response thereto, the system 
presents the selected portion not included within the plurality of segments, and wherein 
after presenting the selected portion not included within the plurality of segments 
presents the selected plurality of segments in temporal order without portions of the 
audio not included within the plurality of segments, and wherein the user selects a 
portion of the audio included within the plurality of segments, wherein in response 
thereto, the system presents the portion of the audio within the plurality of segments 
(page 5, column 1, lines 5-11 and Figure 6). 
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Referring to claim 70, Christel discloses that the temporal information is 
hierarchical and is displayed in such a manner to retain a portion of its hierarchical 
structure (Figure 6). 

Referring to claim 71, Christel discloses that temporal information relates to 
overlapping time periods and the temporal information is displayed in such a manner to 
maintain the differentiation of the overlapping time periods (Figures and page 4, column 
2, lines 26-31). 

Referring to claim 77, Christel discloses a user selectable skip function skips a 
set of frames to a modified location of the video in at least one of a forward temporal 
direction or a reverse temporal direction, and displays the video at the modified location 
(Figure 6). 

Referring to claim 78, Christel discloses a user selectable skip function skips to a 
later temporal segment or a previous temporal segment, and displays video at the later 
temporal segment or the previous temporal segment, respectively (Figure 6). 

Referring to claim 79, Christel discloses that a user selectable scan function 
skips a set of frames to a modified location of the video in at least one of a forward 
temporal direction or a reverse temporal direction, and displays the video at the 
modified location, and thereafter automatically skips another set of frames to another 
modified location of the video in at least one of the fon/vard temporal direction or the 
reverse temporal direction, and displays 
the video at another modified location (Figure 6). 
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Referring to claim 80, Christel discloses that at least one of the fonA/ard temporal 
direction and reverse temporal direction are consistent with the different segments 
(Figure 6). 

Referring to claim 81, Christel discloses that the display of the video is at the 
start of the respective one of the different segments (Figure 6). 

Referring to claim 82, Christel discloses that the display of the video is at a 
predetermined offset within the respective one of the different segments (Figure 6). 

Referring to claim 83, Christel discloses that the graphical user interface displays 
a respective image associated with at least a plurality of the different segments (Figure 
6). 

Referring to claim 84, Christel discloses that the respective image associated 
with the currently presented different segments is visually highlighted (Figure 6). 

Referring to claim 85, Christel discloses that during presentation of the video 
visually highlighted respective images are highlighted in a substantially regular interval 
while the sequence of the presentation of the video is at substantially irregular intervals 
(Figure 6). 

Referring to claim 86, Christel discloses presentation of the different segments 
may be modified by a plurality of different functions, and wherein the user may 
customize another function, not previously explicitly provided, by combing a plurality of 
the plurality of different functions into a single function (page 5, column 1 , lines 9-1 1 and 
Figure 6). 

Response to Arguments 



Application/Contror Number: 10/016,941 Page 19 

Art Unit: 2173 

3. Applicant's arguments, filed 1/30/06, have been fully considered and are 
persuasive. Therefore, the rejection has been withdrawn. However, upon further 
consideration, a new ground(s) of rejection is made in view of Ahmad. The arguments 
related to Christel not disclosing two different types of semantic event data in the 
timelines displayed are persuasive. Ahmad also teaches a media data summarizing 
system obtaining data that is relevant from the media data with different graphic bars 
displaying various types of semantic data, the visual indications being different from 
each other. Both teachings being video summary systems indicate that based on 
teachings of Ahmad, the displaying of visual Indications for two different types of 
semantic event would be obvious. 

Conclusion 

4. The prior art made of record on form PTO-892 and not relied upon is considered 
pertinent to applicant's disclosure. Applicant is required under 37 C.F.R. § 1.1 11 (c) to 
consider these references fully when responding to this action. The documents cited 
therein teach the method for presenting video data. Responses to this action should be 
submitted as per the options cited below: The United States Patent and Trademark 
Office requires most patent related correspondence to be: a) faxed to the Central Fax 
number (571-273-8300) b) hand carried or delivered to the Customer Service Window 
(located at the Randolph Building, 401 Dulany Street, Alexandria, VA 22314), c) mailed 
to the mailing address set forth in 37 CFR 1.1 (e.g., P.O. Box 1450, Alexandria, VA 
22313-1450), or d) transmitted to the Office using the Office's Electronic Filing System. 
Any inquiry concerning this communication or earlier communications from the 
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examiner sliould be directed to Namitha Pillai whose telephone number is (571) 272- 
4054. The examiner can nonnally be reached on 8:30 AM - 5:30 PIVI. If attempts to 
reach the examiner by telephone are unsuccessful, the examiner's supervisor, John 
Cabeca can be reached on (571) 272-4048. All Internet e-mail communications will be 
made of record in the application file. PTO employees do not engage in Internet 
communications where there exists a possibility that sensitive information could be 
identified or exchanged unless the record includes a properly signed express waiver of 
the confidentiality requirements of 35 U.S.C. 122. This is more clearly set forth in the 
Interim Internet Usage Policy published in the Official Gazette of the Patent and 
Trademark on February 25, 1997 at 1195 OG 89. 

Any inquiry of a general nature or relating to the status of this application or 
proceeding should be directed to the Group receptionist whose telephone number is 
(571)272-2100. 

Information regarding the status of an application may be obtained from the 
Patent Application Infomiation Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 

Namitha Pillai ^^-^ — \ { 



Assistant Examiner 
Art Unit 2173 
April 7, 2006 
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